Time-Weighted Dynamic Time Warping Approach for classifying Paddy Rice crops

A GEE implementation. Post-Processing MapBiomas LULC collection

Emanuel Goulart.

2025-08-15

Introduction

DTW for speech recognition and shape matching of words. DTW for speech recognition and shape matching of words.

I stumbled upon Dynamic Time Warping (DTW) during a lecture in Earth Observation. Apparently, in the 80’s, researchers studying speech patterns developed a measure for describing the similarity between two temporal sequences. Although the DTW technique is robust for shape matching, some authors argue that it is not well suited for remote sensing (RS) because it disregards the temporal range when finding the best alignment.

Think of it as each land-cover class having a distinct phenological cycle that is relevant for space–time classification and should be considered for crop monitoring. Therefore, a good time-series land-cover classifier needs to balance shape matching with temporal alignment.

[image with crops classification]

V. Maus and G. R. de Queiroz (2016V. Maus, A. Sanchez, R. Cartaxo, and G. Câmara G. R. de Queiroz. 2016. R: A Time-Weighted Dynamic Time Warping Method for Land-Use and Land-Cover Mapping. https://doi.org/10.1109/JSTARS.2016.2517118.) introduced an improvement by adding a time-weight constraint to DTW. Other developments include treating classification as an object-based rather than pixel-based task, which helps avoid the salt-and-pepper effect. In the RS field, Earth monitoring has been ongoing for many years, and DTW has emerged as a potential tool.

Context

There is no much things on TW-DTW for RS around. The tw-dtw implementation is in R Core Team (2024R Core Team. 2024. R: R. Vienna, Austria: R Foundation for Statistical Computing. https://www.R-project.org/.), which I love, but my laptop seems to struggle on bigger tasks. Necessity pushes us to explore some alternatives and thanks to RS community, there is an implementation of the TW-DTW algorithm in GEE. 1 See Github repository: Willian Oullette

This is a case study of DTW applied for classifying paddy rice crops.

The prime is that tw-dtw performs well without limited training data, making it particularly valuable in data-scarce scenarios. Contrasting with deep-learning (DL) methods which usually requires a higher data training to not under-perform.

MAPBIOMAS

MapBiomas2 See at :MapBiomas is a multi-institutional network providing annual land use land cover (LULC) datasets and monitoring territorial changes, mainly for Brazil. Collection 8, released in 2022, made available a land use mapping of Irrigated Agriculture, classifying irrigation rice crops (paddy rice) in partnership with Brazil’s Water Agency and Embrapa. This collection produces a classification layer and generates annual rice area estimates.

Snippet of MapBiomas panel. Snippet of MapBiomas panel.

The Irrigated Collection, the approach was based on Deep Learning methods using UNet architecture, which has been among the state-of-the-art models for image classification, segmentation, and other tasks. However, DL models, especially for Earth Observation applications, typically demand many training samples.

Brazil’s Rice Heartleand

Location of Santa Catarina within Brazil. Location of Santa Catarina within Brazil.

I was born and raised in this small city where everyone knows each other, surrounded by the sea and facing a river, an outlet of a large basin. Most of my river’s swimming was often alongside vast paddy rice fields that stretched across the entire area. It’s common for paddy rice fields to follow the river channel distribution. Perhaps their full extent can be difficult to gauge from ground level, from ground level, looking from above makes it possible to estimate.

Ararangua hydrographical region. Ararangua hydrographical region.

Through my mostly weekly swimming sessions3 .,the seasons gradually began revealing their patterns, a display of seasonality observed firsthand. At times, the rice fields were a uniform green, covering the parcels like a grassy carpet; other times, they were flooded, attracting birds to the temporary wetlands; and at inter-crop periods, the bare soil was visible. The timing of these seasonal changes in the crops creates a temporal pattern and this ideal scenario for DTW analysis.

Spot for swimming surrounded by the paddy rices. Spot for swimming surrounded by the paddy rices.

Where are talking about southern Brazil, in the Santa Catarina state, home to the small yet big-hearted of Araranguá.

Perhaps what makes this region particularly interesting is its diversity. The landscape transitions from the rugged mountainous plateau in the west, where banana plantations cascade down the hillsides, to the flat coastal plains where rice paddies dominate the river valleys. Urban centers are relatively small. This is not an industrial hub but rather a region whose economy is fundamentally rooted in agriculture. Land use and land cover (LULC) maps provide a clear overview of how the land is actually utilized in the region.

Percentage of land use for each class. Percentage of land use for each class.

The region has a total area of 48,810 km², agriculture represents 54% of the land use, forest represents 32.8% f the area and cities accounts only for 5%. This region represents more than just farmland; it is an economic lifeline, with rice cultivation accounting for 41% of all agricultural activity.

Seasonal patterns

The land seasonality of the region can be well observed when combining different spectral bands from satellite data. The NDVI is a index quantifies the “greenness” of each pixel. 4 NDVI-GIF

Monthly NDVI Monthly NDVI

There is an interesting burst of green during the summer (January), which corresponds to the phenological peak of the rice crops and reflects the full development of the crop canopy. Following flowering, the rice enters the maturation phase, culminating in harvest when the seeds reach their maximum weight—a timing strongly influenced by prevailing weather conditions. After harvest, the NDVI drops, eventually turning completely red, representing the off-season when the land is left as bare soil.

Paddy Rice Spreading

Note that the GIF above do not classify crops, rather it highlights a spetacular seasonal pattern occurring across many pixels.

A classification of paddy rice crops is given by the Irrigation Collection5 Irrigation Collection was based on Unet Deep-Learning method., from MapBiomas. The spatial distribution of the classified paddy rice represents the same patterns observed in the GIF above.

From the observed seasonality, the paddy rice fields exhibit a spatial pattern by spreading extensively across the flat areas of the basin. When it gets towards the west where the mountains are, the land use is identified mostly as forests,

The following map exhibits the paddy rice class extracted from the Irrigation Collection.

Author

Author

TW-DTW

Coming back to TW-DTW, the main strength of this method lies in its lower demand for large numbers of samples. This advantage becomes particularly relevant in regions with scarce data, where deep learning models would otherwise require extensive labeled samples. Fewer required classes also enhance the model’s generalization capacity in such contexts.

Feature Space

Feature space

Feature Description
Sentinel-2 Band 08 NIR (InfraRed)
Sentinel-2 Band 12 SWIR1 (Near Infrared)
Sentinel-2 Derived Index NDVI (Normalized Difference Vegetation Index)
Sentinel-1 VH Horizontal Backscattering
Sentinel-1 VV Vertical Backscattering

Sampling

Training Samples

Feature Number
Rice 13
Non Rice 13

See a explanation about the TW-DTW parameters here: https://github.com/wouellette/ee-dynamic-time-warping/blob/master/dtw.js

Sampling Backscatering and Spectral signatures Sampling Backscatering and Spectral signatures

Parameters of TW-DTW

Parameter Value
Constrain Type Time-constrained
Distance Measure Euclidean Distance
Temporal Range 12 months
Number of Bands 5
Beta - (Time Constraint) 45 days
Alpha 0.1

Results

Classification

The TW-DTW output provides a binary classification of rice and non-rice crops.

Contrast of paddy rice among all other crop types. Author

Contrast of paddy rice among all other crop types. Author

The TW-DTW classification results show that rice crops closely follow the patterns observed in the multi-temporal NDVI composite. However, a noticeable salt-and-pepper effect is present, with many isolated rice pixels scattered across the map. These isolated pixels are likely misclassified, as small-size paddy rice fields are rare due to economic constraints. This highlights the need for post-processing to reduce noise and improve the spatial coherence of the classification.

Dissimilarity Score

Another output of the DTW implementation in GEE is the dissimilarity score, which quantifies the distance between each pixel’s time series and the reference rice time series.

Author

Author

These dissimilarity scores represent the distance between each pixel’s time series and the reference rice time series. Generally, lower scores indicate higher similarity to the target class, whereas higher scores suggest that the pixel is less likely to belong to that class.

Dissimilarity histogram of Rice Dissimilarity histogram of Rice

In the dissimilarity map, paddy rice fields typically have scores ranging from 0 to approximately 20,000. Beyond this threshold, the number of pixels with high dissimilarity scores decreases substantially, indicating fewer samples with low similarity. This demonstrates that the TW-DTW algorithm effectively leverages these scores for classification, and thresholding provides a deterministic way to fine-tune the classification by separating likely rice pixels from others.

The dissimilarity scores for rice pixels also exhibit a lower standard deviation compared to non-rice pixels, reflecting less variability within the rice class. The rice histogram shows a pronounced tail and a relatively small secondary peak around a dissimilarity score of 60,000, a feature absent in the non-rice histogram. Although non-rice scores display greater variability, they are confined to a narrower range compared to rice.

Assessment

Jaccard Index Jaccard Index

Dice Coefficient Dice Coefficient

To assess the results, Jaccard Index, Overlap and Dice coeffient.

Similarity of TW-DTW predictions with MapBiomas Irrigation

Asset Area (km2) Jaccard Overlap Dice Absolute Error (km2)
Irrigation MapBiomas 9029.62 1.0000 1.0000 NA NA
TWDTW 8583.86 0.8199 0.8626 0.84 445.76
OTSU Threshold 8306.36 0.8184 0.8896 0.85 723.26
Arbitrary Threshold 8521.98 0.8196 0.8683 0.84 507.64

GEE Panel

Access the panel at GEE Panel The results are fully accessible on the GEE panel, where you can compare the TW-DTW predictions with the MapBiomas irrigation collection and other land cover classes.

ACCESS THE PANEL

It is also possible to explore a multi-temporal composition for visualizing some of some band values (e.g: NDVI, VV, NIR).

Multi temporal composition. Red pixels are June, Green pixels are January and the Blue band represents March.

Multi temporal composition. Red pixels are June, Green pixels are January and the Blue band represents March.

This following image shows a as a red green band the month of January (coidning with the peak of rice crop), red band the month of June (coinciding with the barren soil) and blue in March.

Panel exhibiting the results of the current study. Panel exhibiting the results of the current study.

If you not well in the gif, you will able to see that a very distinctive peak of green in March also happens, which denote crops types that are yielded before the winter. Which shows up this bluetish pattern on certain areas of the multi-composition

Panel in GEE with the multi-temporal asset available.. Panel in GEE with the multi-temporal asset available..

Considerations

TWDTW shows a potential for operational crop monitoring, particularly in regions with limited availability of training data. Its ability to achieve competitive accuracy with substantially fewer samples makes it a valuable tool for agricultural monitoring in data-constrained environments.

However, several limitations must be acknowledged. TWDTW is computational expensive, which restricted the analysis to a single year, limiting the evaluation of interannual variability.

There is also a potential bias from wetland areas. It seems that the signal on wetlands has some degree of proximity with rice fields. It may come from the surface water being higher during flooding periods.

Further Development

While TW-DTW presents promising capabilities, it also exhibits vulnerabilities that leave considerable room for improvement. In this study, only polarimetric bands VV and VH, along with optical bands SWIR1 and NIR and the NDVI index, were used for inference. Alternative band fusion approaches could be explored, including indices designed to better preserve temporal feature characteristics. It remains unclear whether adding other bands or indices would enhance temporal similarity scores and improve classification performance.

Future developments should integrate radar-based scattering mechanisms related to crop development—such as polarimetric decomposition features, entropy, anisotropy, embedding, and angle measurements—to enrich the temporal information. The dissimilarity score could also be exploited as a temporal tensor or incorporated into hybrid architectures, where TW-DTW serves as a feature within deep learning models.

Acknowledgements

The Remote Sensing community is very alive, pushing for more open source frameworks and tools. Thanks to:

1 - MapBiomas - For the incredible work developed in Brazil.

2 - Soil Watch - For their work and amazing GEE repositories.

3 - Willian Oulette For implementing twdtw on GEE

4 - All my teachers and colleagues for their inspiring work. Prof. Dr. Dirk Tiede;

Thank you

Emanuel Goulart Farias.

Master’s student at Copernicus Master in Digital Earth and Geodata Science. This study was conduct as part of a discipline - Analysis and Modelling - within Paris-Lodron University of Salzburg.